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Abstract —Fountain codes like LT or Raptor codes, also 
known as rateless erasure codes, allow to encode a message 
as some number of packets, such that any large enough 
subset of these packets is sufficient to fully reconstruct 
the message. It requires undamaged packets, while the 
packets which were not lost are usually damaged in real 
scenarios. Hence, an additional error correction layer 
is often required: adding some level of redundancy to 
each packet to be able to repair eventual damages. This 
approach requires a priori knowledge of the final damage 
level of every packet - insufficient redundancy leads to 
packet loss, overprotection means suboptimal channel rate. 
However, the sender may have inaccurate or even no 
a priori information about the final damage levels, for 
example in applications like broadcasting, degradation of 
a storage medium or damage of picture watermarking. 

Joint Reconstruction Codes (JRC) setting is introduced 
and discussed in this paper for the purpose of removing 
the need of a priori knowledge of damage level and sub¬ 
optimality caused by overprotection and discarding under¬ 
protected packets. It is obtained by combining both pro¬ 
cesses: reconstruction from multiple packets and forward 
error correction. Intuitively, instead of adding artificial 
redundancy to each packet, the packets are prepared to 
be simultaneously payload and redundancy. The decoder 
combines the resultant informational content of all received 
packets accordingly to their actual noise level, which can 
be estimated a posteriori individually for each packet. 
Assuming binary symmetric channel (BSC) of e bit-flip 
probability, every potentially damaged bit carries Ro (e) = 
1 — hi{e) bits of information, where hi is the Shannon 
entropy. The minimal requirement to fully reconstruct the 
message is that the sum of rate Roie) over all bits is at 
least the size of the message. We will discuss sequential 
decoding for the reconstruction purpose, which allows to 
work close to the theoretical limit. Analysis and tests of 
the accompanied implementation show that the statistical 
behavior of constructed tree can be approximated by 
Pareto distribution with coefficient c such that the sum 
of Rc{c) = 1 — /ii/(i+c) (e) is the size of the message, where 
huie) = (e“ -I- (1 — e)“)/(l — u) is u-th Renyi entropy. 

Keywords: fountain codes, error correction, sequen¬ 
tial decoding, Renyi entropy, broadcasting 

I. Introduction 

Many communication settings require dividing a po¬ 
tentially large message (payload) into relatively small 
blocks of data, which will be referred as packets. These 
packets are often damaged or even lost on the way. Both 
issues could be handled by allowing for retransmission 
of uncertain or missing parts of data. However, it would 


message (payload); 



Figure 1. Schematic diagram of different possibilities of splitting 
the informational content of the message. 1) Simple split requires all 
undamaged packets. 2) For fountain codes, any large enough subset 
of undamaged packets is sufficient. 3) While the set of packets is 
already redundant, we can add additional redundancy to sepai'ately 
protect every packet - some turn out overprotected, some insufficiently. 
4) JRC allows to extract the actual informational content (green area) 
from all obtained packets. Reconstruction requires to gather a sufficient 
amount of informational content: such that the sum of 1 — h{e) over all 
bits exceeds the size of the message, where e is the bit-flip probability 
of assumed binary symmetric channel (BSC). 


require costly bidirectional communication, waiting for 
packets before classifying them as lost (automatic repeat 
request after a timeout). Hence, there is a strong need 
for methods allowing to reduce or completely eliminate 
the feedback, schematically presented in Fig. [T] 

The issue of missing packets can be handled by 
various erasure channel methods, for example by the 
fact that coefficients of degree d polynomial can be 
obtained from values in any d+1 points in classic Reed- 
Solomon codes m.A faster way, pioneered in LT codes 
by Michael Luby ^ in 2002, is constructing the packets 
as XOR of some subsets of data blocks, usually chosen 
in a random way. This approach is currently referred 
as fountain codes(FC) or rateless erasure codes and was 
improved to linear encoding and decoding time in Raptor 
Codes 0. There are considered various applications of 
FC like relay networks sensor networks Q, radio 
networks ijb), networked storage 111 or watermaring 0. 

If a packet was not lost, it is often received damaged. 
Forward error correction (FEC) layer is used to handle 
this issue without retransmission: the sender attaches 
some redundancy, which will be used by decoder to 
repair eventual damages. The applied redundancy level 
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determines a statistical threshold for damage, deciding 
if a given packet can be repaired. Fountain codes do 
not tolerate damaged packets, so those exceeding the 
damage threshold need to be discarded - their entire in¬ 
formational content is being lost. As this threshold has a 
statistical nature, this all-or-nothing means inefficient use 
of the channel: some packets turn out to be overprotected 
- have used an unnecessarily high amount of redundancy, 
some packets insufficiently - loosing all the content. 

Additionally, in many scenarios the sender has inaccu¬ 
rate or no a priori knowledge of the final damage level, 
hence some overprotection needs to be used, and some 
of packets are still being lost. For example 

• broadcasting: while every packet (receiver) may 
have a different individual damage level, the sender 
needs to prepare universal packets, 

• protecting storage media: the final damage level 
depends on age, conditions and random accidents, 

• sending a packet through a network: there is often 
no knowledge of the route it will take, 

• embedding a (robust) watermaring: there is no 
knowledge of the final capturing conditions, 

• in some scenarios the noise level can vary too 
quickly for adaptive change of redundancy level, for 
example in acoustic or radio communication (e.g. 
underwater). 

This damage level can be usually estimated a posteriori - 
correspondingly: basing on the conditions for individual 
packets, history of given storage medium, on the actual 
route of a given packet, on the parameters of captured 
image, on more recent evaluation of environmental 
parameters. Moreover, the applied redundancy could 
allow to estimate the actual damage level from the 
received packets. Therefore, it would be beneficial to 
allow to shift the need of knowledge of damage level 
from a priori to a posteriori. 

Joint Reconstruction Codes (JRC) setting is intro¬ 
duced, analyzed and discussed for this purpose. While 
offering the same rates, they require only a posteriori 
knowledge of the damage level thanks to combining 
both processes: of error correction and of reconstruction 
from multiple packets. The sender prepares packets 
to be treated simultaneously as the message (payload) 
and FEC redundancy. The decoder searches the space 
of promising candidates for the message sequence, to 
get the best agreement with all the received packets 
accordingly to their a posteriori damage level - individual 
level of trust for each packet. 

For simplicity we will only consider the binary sym¬ 
metric channel (BSC) here: that every bit has inde¬ 
pendent e probability of being flipped. Theoretical rate 
limit in this case is i?o(e) := 1 — h{e), where h{p) = 
—p\g{p) — (1 — p)lg(l — p) is the Shannon entropy. 
Therefore, the minimal requirement for reconstructing 
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Figure 2. Rate: informational content per bit assuming binary symmet¬ 
ric channel (BSC). Renyi entropy hu{e) = lg(e^ + (1 — e)^)/(l — w) 
becomes Shannon entropy for u ^ 1 (c ^ 0). The c = 0 rate is the 
theoretical bound. The remaining cases allow to find Pareto coefficient 
c describing statistical behavior of the correction process: such that 
sum of Rc{^) over all bits is the size of the message. 

a message is that the sum of 1 — h{e) over all received 
bits is at least the size of the message. It is schematically 
presented in Fig. [T] using green bars containing 1 — h{e) 
of size of the packet - the minimal requirement for 
reconstruction is that the green bars sum to the length 
of the message. 

Practical error correction methods usually work be¬ 
low this theoretical rate. We will discuss implemented 
enhancement of sequential decoding, for which there 
is known so called cutoff rate 13 . This article uses a 
generalized family of rates described by c > 0 parameter 
(Pareto coefficient) as in Fig. 

^c(e) = 1 —/ii/(i-i-c)(e) (1) 

where huie) = ^ Renyi entropy ifTOl . 

Ig = log 2 . The u = 1 case is naturally defined as the 
M —1 limit, getting the Shannon entropy: 

hi{e) = h{e) = -elg(e) - (1 - e) lg(l - e). (2) 

Finally, we have the two mentioned special cases: 

Roi^) = 1 ~ h{e) 

^i(e) = 21g(v^+Vl-e) = Ig + 2v/e(l - e)) 

Rq is the theoretical rate limit for this channel. i?i is the 
cutoff rate for sequential decoding below this rate, 
sequential decoding of infinite message would require 
a finite width - average number of steps per bit block. 
Generally, as it will be shown in Section |IV| c is the 
coefficient of Pareto distribution describing statistical 
behavior of the correction process: while increasing 
twice the maximal number of considered steps, prob¬ 
ability of failure drops asymptotically 2'^ times. Thanks 
of using finite data frames and some improvements to 
the sequential decoding: large internal state (64 bit) and 
bidirectional correction, much larger rates than for the 
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c — 1 cutoff rate can be used JTTl . e.g. for c = 1/2 
and IkB data frames there were still obtained complete 
correction in all tests for all three considered cases: 7/8, 
1/2 and 3/4 rate. 

Finally, the c value describing correction process is 
such that the sum of i?c(e) over all bits is the size 
of the message. The cost (difficulty) of correction 
statistically decreases with the growth of c. Therefore, 
from JRC perspective we should choose some boundary 
value: Cmin and wait (gather packets) until the sum of 
(e) over the received packets exceeds the message 
size. Then the decoder can try to perform the correction 
and eventually wait for another packet(s) to try again 
if it has failed to reconstruct within assumed resource 
limit (time and memory). For unidirectional sequential 
decoding (implemented) this Cmin boundary can be 
chosen in « [0.5,1] range, depending on the resource 
limit. Bidirectional sequential decoding effectively 
allows to halve this value. 

Let us briefly look at some possible applications of 
this general possibility of removing the need of a priori 
knowledge of damage level: 

* A basic example is broadcasting, where the sender 
have to use universal packets. A posteriori adapta¬ 
tion to the actual noise levels of individual packets 
(receivers) may be beneficial, and can be effectively 
obtained by JRC. 

• Another basic application is protection of storage 
media. For example imagine a thousand of DVD 
copies of a movie. While time passes, all of them 
will degrade. Anally exceeding the included pro¬ 
tection level - making all of them useless. JRC 
approach would allow to still reconstruct the orig¬ 
inal content from some number of badly damaged 
copies, where the required number depends on dam¬ 
age level. As discussed in Section m a message 
encoded this way can be prepared for 3 levels of 
decoding: fast straightforward decoding from any 
single undamaged disc, or with error correction 
for a lightly damaged single disk, or with error 
correction for multiple heavily damaged disks. 

* Another family of applications is improving effi¬ 
ciency and reducing energy consumption of various 
networks by replacing error correction applied by 
every node, with a single reconstruction-tcorrection 
applied by the receiver only (of the payload - some 
small headers need to be frequently corrected). 
Decreasing energy consumption of network nodes 
is especially important for battery operated, e.g. 
sensor networks. 

• For watermarking applications we could divide a 
message into small packets - the receiver should 
be able to reconstruct the message even if some 
of packets are missing (e.g. part of a picture or 


frames of a video) and the remaining are damaged 
accordingly to capturing conditions - in an unknown 
to the sender way. 

• This approach can be also used when there are no 
missing packets (some additional optimizations can 
be applied in this case). For example imagine acous¬ 
tic or radio communication while rapidly varying 
environmental conditions - the receiver can likely 
receive all the packets, and he usually has more 
information about the actual damage level than the 
sender, making JRC beneficial. 

The situation with asymmetric sender-receiver knowl¬ 
edge is considered for example in the Kuznetsov- 
Tsybakov problem Ha, where only the sender knows 
the pattern of fixed bits of the channel. In contrast, we 
will assume here that the receiver have more information 
about the noise levels of individual channels - we will 
refer to it as unknown noise level problem (UNL prob¬ 
lem). As expected, in both problems we can approach 
rate as if both sides would have the missing information. 

The article is constructed in the following way. The 
JRC error correction setting will be formulated in Sec¬ 
tion]^ There will be introduced the considered encoding 
scheme and separate decoding procedures in two cases: 
for undamaged packets only and for the general case 
(with included error correction). The undamaged case 
will be discussed in Section III - both theoretically and 


comparing with experimental tests. This section begins 
with straightforward decoding: when we need to consider 
only a single candidate. Then a general undamaged case 
is discussed, where it turns out that we need to consider 
only a small number of candidates per step (< 2 on 
average). Section IV contains theoretical analysis and re¬ 
sults of experiments for the general damaged case, where 
the number of candidates per step is approximately 
described by Pareto distribution with coefficient found 
using Renyi entropy. Some idealized UNL problem will 
be analyzed in Section to understand performance 
of different possibilities, especially the concatenation of 
FC and JRC. Finally, Section |Vl] summarizes the new 
possibilities of JRC coding paradigm and discusses some 
further research questions and improvement opportuni¬ 
ties. 


11. Joint Reconstruction Codes (JRC) 

We will now formulate the error correction setting and 
briefly introduce coding/decoding methods. Details can 
be found in the accompanied implementation lfT3l . The 
presented approach is expansion of the method described 
in im, where more discussion can be found, especially 
for bidirectional correction. 

A. Encoding 

Let us start with the fountain code (FC) situation. 
Assume the size of the message is NL and we are 
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Figure 3. Two approaches for combining the possibilities of fountain codes with error correction (two ”X”-marked packets have been lost) for 
M = 3 received packets. Left: correct each packet independently, then use FC. Right: JRC tries to reconstruct successive bits of the message 
from bits on the con'esponding position of simultaneously all packets. 


receiving some M packets of length L bits from a larger 
set of packets. The minimal requirement for reconstruc¬ 
tion is M > N. We will now allow the packets to 
be damaged, as depicted in Fig. For simplicity there 
will be assumed binary symmetric channel (BSC): every 
bit of packet i G [1,AT] has probability of being 
flipped. If we would apply forward error correction for 
(a priori) known e values, the minimal requirement for 
reconstruction is ~ ^ We will get the 

same bound for JRC without the a priori knowledge. 

Let us divide the message into length N bit blocks, 
each block corresponds to a single bit in every packet 
(vertical lines in Fig. |^. Specifically, the encoding 
procedure has L steps. In A:-th encoding step (k G 
{0, ... ,L — 1}) there are used {Nk, ...,N(k -f 1) — 1} 
bits of the message (bit block x), to produce /c-th bit 
of every packets. There is required an internal state of 
encoder to connect redundancy of successive blocks. 
The current implementation uses 64 bit state for this 
purpose, producing fc-th bit of i-th packet as i-th bit of 
the state. This way the number of produced packets is 
limited to 64, the actually received packets correspond 
to some subset of these 64 bits. The internal state needs 
to be modified accordingly to the currently encoded N 
bit block X. There is used a pseudorandomly chosen 
transition function / : {0,..., 2^ — 1} —{0,..., 2^* — 
1} for this purpose. The state transition is a cyclic 
shift of (state XOR f[x]). The encoding procedure is 
schematically presented as Method [T] example of its 
application is presented in Fig. Observe that using 
a pseudorandom number generator (PRNG) initialized 
with a cryptographic key to choose the / function, we 
could include encryption in such encoding. 

Finally encoding procedure is: set state as some 
arbitrary initial_state (known to receiver), then perform 
encoding step for blocks 0 to L — 1. The final state: 
final_state would be beneficial for decoder for the final 
verification of unidirectional decoding, and is necessary 
for bidirectional decoding. It may be included in the 


header of packet. However, unidirectional correction can 
be performed without it, at cost of probable damage of 
some last bits of the message. 

Algorithm 1 Encoding procedure 

Require: state = initial_state 
for fc = 0 to L — 1 do 

set X as fc-th N bit block to encode: 

X = {Nk, ..., Ai(fc -b 1) — 1} bits of the message 
state = state XOR f[x] {state transition} 

for all i, set fc-th bit of packet i as i-th bit of state'. 

packet[i]^. = statei {xi is f-th bit of x} 
state = state >> 1 {cyclic shift by one position} 
end for 

final_state = state {it can be useful for decoding} 


B. Decoding for undamaged case - no error correction 

The decoding procedure searches for a message lead¬ 
ing to all the received packets by expanding a tree of 
promising candidates. Depth k node of this tree repre¬ 
sents a candidate of length Nk prefix of the message, 
corresponding to k length prefixes of all the received 
packets. Expanding a node denotes creating its children 
corresponding to N bit block longer prefix of the mes¬ 
sage. 

In the undamaged case we consider only candidates 
having complete agreement with all the received packets, 
the remaining expansions are not considered. In this 
case, a natural approach is to consider a list of all 
candidates up to a given position and successively shift 
this position. The candidate list for the next position is 
obtained by expanding all candidates from the previous 
list by a single step, considering only those being in 
agreement with the corresponding bit of all received 
packets. 

Eor performance reasons, this method should have 
prepared a list of allowed single step expansions for all 
cases. We have received some M of e.g. 64 packets. 
Denote which\i] G 64} for i = 1,..., M as the 

original number of i-th received packet (its bit position). 
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Figure 4. Example of encoding process for 5 bit state and 5 packets, N = 2 size bit blocks, M = 3 received packets and /[OO] = 
01100,/[Ol] = 10010,/[lO] = 01010,/[ll] = 10110 transition function, chosen in a pseudorandom way. The received packets are e.g. 
which[l] = 1, which[2] = 3, which[3] = 4, the extract : {0,..., 31} {0,..., 7} function returns 1,3,4 bits of 5 bit state here. 

Symbol © denotes XOR, ”>>” denotes cyclic shift right by 1 position of the 5 bit state. 


Denote by data[k] the M bit sequence of /c-th bit of all 
received packets; 

data[k]i = receivedjacket[i\k 

for j = 1,...,TO, k = {),... ,L — where Xi denotes 
*-th bit of X here. Denote by extract {state) as M bit 
sequence of bits of state on positions corresponding to 
all received packets; 

extr act {state) i = state^hichli] for i = 1,..., M 

Now a node can be expanded by sequence x if data\k\ = 
extract{f\x\ XOR state), or in other words if 

extract{f[x]) = extr act {state) XOR data[k] ;= z 


Hence, the current situation can be described by z which 
will be referred as a syndrome. We can have prepared 
candidates[z\ table containing a list of x being in 
agreement with syndrome z G {0,..., 2^ — 1}. The 
decoding method is schematically presented as Method 
Its performance is discussed in Section III 


Algorithm 2 Decoding for undamaged case 

Require: current_list = {initial_state} {root of the tree} 

Require: candidates\z]: list of all x such that extract{f\x\) = z 
for fc = 0 to L - 1 do 
empty new_list 

for every node in current_list do 

end = candidates[extr act {node, state) XOR dafa[/c]] 
append end with updated states to new_list 

end for 

current_list = new_list 

end for 

trace back any coding from the final new_list (with state = 
final_state if available) 


For each of 2^ syndromes z, there are possible 
2^ candidates, each of them has 2~^ probability 


of agreeing with all the received packets. Hence, the 
expected size of the list pointed by candidates[z] is 
2^/2^. Finally, the total size of all lists of candidates 
and their pointers is 0{2^ + 2^) here. This exponential 
growth restricts to reconstruction only from a relatively 
small M. It could be slightly relaxed for example by 
splitting the M packets into a few subsets and use a 
separate candidates table for each of them, intersecting 
their lists. 

Interleaving. A more efficient way for increasing the 
number of packets for reconstruction is splitting the 
decoding step into S' G N substeps (phases), which use 
disjoined subsets of packets. These substeps should be 
performed cyclically, hence we will refer to this ap¬ 
proach as interleaving. The phase of encoding/decoding 
is indexed by 0 < s < S number of substep, and s 
increases cyclically modulo S in successive steps. 

We would like that such S successive substeps cor¬ 
respond to a single original step - all further analysis 
should be seen that, in the case of interleaving, a single 
step is in fact grouped S successive substeps. Let us 
split the original N bit blocks of the message per step, 
into bit blocks used in separate phases; N = X)s 
(Ns G N), which can be equal. The encoding Method 

corresponds to a single substep now, evolving the 
same state in succeeding phases. In phase s it uses 
successive Ng bit block of the message to produce 
successive bit of s-th subset of packets. Hence, in the 
current setting, it could produce 645' packets. If all 
Ns are equal, we can use the same transition function 
/ ; {0,..., 2^»} ^ {0,..., 2^4 - 1} for all phases. 

The decoding analogously goes cyclically through 
these 5 phases, considering all possible candidates up 
to position corresponding to a given substep. The re- 
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ceived packets are now from S subsets: M — 
where Mg is the number of received packets from s-th 
subset. Now we need two-dimensional candidates[s][z] 
table, containing all possible candidates for the current 
phase (s) and syndrome (z). Analogously to the non¬ 
interleaving case, we can estimate the memory require¬ 
ment for all lists and pointers as O + 2-^=^ 

This formula is minimized for uniform split of both N 
and M. The former can be obtained by choosing all 
Ng equal. Uncontrolled nature of received packets might 
be an issue. We could enforce upper bound for Mg by 
limiting the number of packets from a single phase, or 
eventually use a few candidates tables for a given phase 
and intersect their lists. 

C. Decoding in the general case (with error correction) 

In the general case we allow for damaged packets: 
tolerate some disagreement with the received packets 
while reconstruction. The higher assumed (a posteriori) 
damage level (e^), the higher tolerance for disagreement 
of bits of a given packet. Considering all candidates up to 
a given position, like in Method would be completely 
impractical here as their number would grow exponen¬ 
tially. Instead, in every step we will find and expand 
only the most promising node, as the one maximizing 
weight W, which is logarithm of the probability of 
being the proper node, obtained using Bayesian analysis. 
As discussed in HD, this probability is a product of 
probability of the assumed error vector and one over 
probability of accidental agreement of incorrect nodes. 

Assuming that a given node is incorrect, the probabil¬ 
ity of accidental agreement of some of its 2^ possible 
expansion with M bits of received packets is 2^/2^. 
Let us denote A:-th step error vector for a given message 
candidate by = {E^}. = 0 if fc-th bit of i-th 

packet agrees, 1 otherwise: 

E* = extract(state XOR f[x]) XOR data[k] 

Now assuming that Ath packet came through BSC, 
probability of E^ error vector is: 

M 

Pr(E'=) = (3) 

i^l 

where Pe(0) := 1 — e , Pe(l) := e 

We can finally write the formula for weight we want to 
maximize: 

W = Y^W{E.^) 

k 

M 

WiE’^) = M - N+ '^\g{Pg,(E^)) (4) 

i=l 

In a given moment, the most probable node to expand is 
the one having the largest weight W. The M — N term 


favors longer expansions, the error vector term contains 
a penalty for disagreement with the received damaged 
packets. The weight of expansion of a node is the weight 
of its parent plus lU(E^) of the currently considered 
expansion, evaluating disagreement with A:-th bit of the 
packets. 

In practical applications we need to quickly find 
the best candidates for given expansion. As for the 
undamaged case, the current situation is sufficiently de¬ 
scribed by syndrome z = extract{state) XOR data[k]. 
This time candidates[z] lookup table should contain 
2^ bit block candidates x, sorted by the W{z XOR 
extract{f[x\)) weight. The final decoding procedure is 
schematically presented as Method detailed imple¬ 
mentation is in m, its performance is discussed in 
Section |IV] 

Algorithm 3 Sequential decoding for the general case 

Require: empty heap of nodes to consider 
Require: candidates[z]: list of 2^ blocks x 

sorted by W{z XOR extract(f[x])) 
insert root to heap: initial_state, k = 0 position, W = 0 weight 

repeat 

node = getMa.x(heap) {retrieve max weight node from heap} 

if applicable, add to heap 

the next of candadates of parent of node 
add first of candidates of node to heap 
until reached final position (and if available: state = final_state) 


This time candidates table has much larger memory 
requirements: 0{N ■ 2^^^) as we need all sorted x G 
{0,..., 2^ — 1} for every syndrome z G {0,, 2*^—1}. 
There could be used approximated scheme to slightly 
relax the restriction for the number of packets for recon¬ 
struction. For example, to approximate the optimal order, 
one could use candidates table only for some number 
of the least damaged of received packets (as they have 
the highest impact on the weight). Then all the received 
packets can be used to calculate the weight of a given 
extension. 

Using interleaving, the total memory cost is analo¬ 
gously O • 2^^“+^=). As previously, Ng should 

be chosen equal, while random nonuniform split of M 
might be an issue, limiting the maximal number of 
received packets from a single phase used for recon¬ 
struction. As in the non-interleaved case, if exceeding 
the assumed resources, the least damaged should be used 
to approximate the optimal order. 

III. Undamaged packets case 

Let us now analyze the situation with only undamaged 
packets - analogous to fountain codes, no error correction 
included. We can use simpler decoding Method]^ in this 
case: consider all candidates up to a given position and 
successively shift this position. 
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Table I 

Probability of straighforward decoding for random transition function (eq. 


N 

M = 1 

M = 2 

M = 3 

M = 4 

M = 5 

M = 6 

M = 7 

M = 8 

1 

0.5 

0.75 

0.875 

0.9375 

0.96875 

0.98436 

0.99219 

0.99609 

2 

- 

0.09375 

0.41016 

0.66650 

0.823059 

0.0.90891 

0.953794 

0.97673 

3 

- 

- 

0.00240 

0.12082 

0.38572 

0.634028 

0.79999 

0.89542 

4 

- 

- 

- 

1.1 ■ 10-® 

0.01040 

0.12901 

0.37613 

0.61971 

5 

- 

- 

- 

- 

1.8 ■ 10-13 

7.6 ■ 10-5 

0.01442 

0.13236 

6 

- 

- 

- 

- 

- 

CO 

o 

1 

to 

“4 

4.2 ■ 10-l> 

0.00018 

7 

- 

- 

- 

- 

- 

- 

7- 10-55 

1-10-17 

8 

- 

- 

- 

- 

- 

- 

- 

3 ■ 10-11° 


A. Straightforward decoding 

Let us look at {extract{f[x])}^^Q 2 N_i length M 
sequences; containing bits of f[x] corresponding to all 
possible bit blocks of the message (x). Observe that if all 
these sequences turn out different, the block of fc-th bit of 
the received packets (data[k]) immediately determines 
the unique x to decode; candidates[extract{state) 
XOR data[k]] is a single value. We will refer to this 
case as straightforward decoding case; we can directly 
determine the only possible candidate, move to succes¬ 
sive position and so on as in Method The number of 
decoding steps is exactly the number of encoding steps 
in this case. 


Algorithm 4 Decoding for straighforward case 

Require: state = initial_state 

Require: candidates[z]: the only x such that extract{f[x]) = z 
for fe = 0 to L — 1 do 

X = candidates[extract{node.state) XOR data[k]] 
produce x {decoded symbol} 

state = (state XOR f[x]) << 1 {cyclic shift} 

end for 


Hence, assuming a completely random transition func¬ 
tion / (Pr(0) = Pr(l) = 1/2), we can find probability 
of such straightforward decoding; that 2^ random length 
M > N sequences will be different. Table [I] contains 
some its numerical values. 

(2M)!/(2M_2iV)! 

Pristraighforward decoding) = - — 

(2M)(^ ) 

(5) 

Assuming a completely randomly chosen transition 
function /, for some parameters like M » N we 
can get straightforward decoding with a reasonable 
probability. 

In many scenarios we could improve this probability 
by designing transition function and influencing the 
subset of packets used for reconstruction. Let us discuss 
such possibility for the following practical example; 

Storage media protection problem; we would like to 
store a message on a few storage media like DVD disks, 
such that we can retrieve the message independently 
from any single one of them as long as there is no 
damage, or it is low. Additionally, for high damage, we 


should be able to retrieve the message from a few of 
these storage media. 

For speed purposes, let say we would like to straigh¬ 
forward decode N = 8 bits/step from such single 
undamaged storage medium. It can be done by storing 
8 IRC packets on such single medium, and choose 
the transition function as a (random) permutation of 
bits corresponding to these packets. In other words, 
extract[f (x)] : {0, ...,255} —{0, ...,255} should 
be chosen as a permutation for extract choosing bits 
from a single medium. This way we get straighforward 
decoding from such 8 undamaged packets. If we want to 
be able to repair some damage for such single medium, 
we can place more of these packets there. 

For example imagine recording 4 disks, using 16 
(of 64) different packets for each of them. Now for 
the 8 undamaged packets of any of these disks we 
can straightforward decode the message. Using all 16 
packets from one medium, we get rate 1/2 forward error 
correction (maximum bit-flip probability e « 0.110). 
Using 64 packets from all 4 disks, we get rate 1/8 
forward error correction (maximum bit-flip probability 
e « 0.295). Hence, if encoding a message this way, 
IRC allows here for 3 levels of decoding depending on 
available resources. 

B. General undamaged case 

Let us define width, characterizing the decoding pro¬ 
cess as the average number of candidates considered per 
position 

number of decoding steps 
width = — 

number of encoding steps 

^ nodes considered while reconstruction 

^ z 

The number of encoding steps for LN bit message 
is L. The number of decoding steps is the number of 
considered candidates; nodes of reconstruction tree. 

For straightforward decoding width = 1, which is 
the lower bound. We are interested in understanding the 
probability distribution of width for given reconstruction 
parameters. As it will be explained, we should expect 
approximately Gaussian distribution for the undamaged 




















M > N case, what is confirmed by tests. For dam¬ 
aged case: with included error corTection, we get ap¬ 
proximately Pareto distribution of width, what will be 
discussed in Section EY] 

Figure shows widths for M = N + 1 cases and 
N = 1,..., 8. It was obtained by performing 1000 re¬ 
constructions for random conditions (message and subset 
of packets) using Method and then sorting obtained 
widths. If we would switch both axes of this plot 
and normalize, we would get empirical approximation 
of the cumulative distribution function for width. In 
its left panel (low N) we can see some width = 1 
lines corresponding to straightforward decoding - up 
to approximately 0.75, 0.41, 0.12, 0.01 of 1000 for 
correspondingly N = 1,2,3,4, as we can read from 
Table U 

For a larger N we can see a shape resembling cumula¬ 
tive distribution function of Gaussian distribution. Let us 
find this behavior assuming some randomness. Imagine 
that in a given step we have considered i candidates. 
They have 2^i possible expansions - one of them is 
the proper message, the remaining have 2~^ probability 
of accidentally agreeing with M bits from the received 
packets. Finally, the probability that there will be j 
candidates to consider in the next position is 

(1 - 2-^f^-^ (6) 

This stochastic matrix (of infinite size) allows to find 
the stationary probability distribution for the number of 
candidates to consider: such that pi = PjPij where 
Pi is the probability that there will be i candidates in a 
given position. This {pi}“i distribution tends to 0 for 
M > N (not true for M = N). In the M = iV -f 1 case 
it stabilizes for large N on approximate distribution: 

{0.41884,0.32221,0.15680,0.06446,0.02438, 

0.00875,0.00303,0.00102,0.00034,0.00011,...} 

with expected value 2 and standard deviation 3.28. 
For M = N + 2 case the asymptotic distribution is 
approximately: 

{0.72383,0.22726,0.04175,0.00620, 
0.00084,0.00011,0.00001,...} 

with average value 1.33 and standard deviation 2.58. 

Finally, for the M = N + 1 case and N >« 4, the 
width of Method should be approximately a Gaussian 
distribution with expected value 2. It means that per 
position there is 1 proper candidate and on average 1 
improper. However, Figure. clearly shows expected 
value 1.5 - the difference is caused by using Method 
[^for these results. This algorithm considers candidates 
of various lengths, favoring the longest ones in the 
undamaged case. Hence, after considering the proper 


candidate, it will not consider the remaining candidates 
on a given position here. The proper candidate is on 
average in the middle, hence the expected width for 
Method should be 1 (proper) + 0.5 (improper) =1.5 
here, what agrees with the Figure. 

One could wrongly conclude that it is better to use 
Method 1^ for the undamaged case to reduce the lin¬ 
ear coefficient e.g. from 2 to 1.5 here. However, this 
algorithm requires finding the best node for each step, 
making it more costly than Method]^ 

C. Conclusions and comparison with fountain codes 

In this section we have discussed proposed coding 
approach as a direct replacement for fountain codes: 
for undamaged packets. For some probability, M = N 
packets are sufficient to fully reconstruct the message, 
especially for small N. For the M — N + \ case: one 
excess packet, the decoding is very cheap: requires on 
average twice more steps than encoding. Probability of 
failure is practically zero. 

In contrast, FC have relatively large probability of 
failure for a small number of excess packets (the used 
random matrix is not invertible), like « 50% for the 
M = N + l case lfT4l . However, they can inexpensively 
operate on arbitrarily large number of packets, what is 
costly for JRC approach. 

Another advantage of FC is retrieving a part of 
the message if complete reconstruction is not possible. 
This subset is usually random, however, more impor¬ 
tant packets can be better protected in unequal error 
protection EB variant of FC. In contrast, JRC tries to 
reconstruct the entire message. If it fails: exceeds the 
assumed resources, the beginning of the message up to 
this position (and the end in bidirectional correction), are 
nearly properly reconstructed. 

IV. General damaged case 

We will now discuss the performance of the general 
case: with allowed damaged packets. This time decoding 
is much more costly due to included error correction. 
While for undamaged case the width had approximately 
Gaussian distribution with a small expected value (« 
2 for the M = N + 1 case), the width for damaged 
case is approximately described by Pareto distribution 
with parameter c > 0: increasing twice the width limit 
decreases probability of failure approximately 2'^ times. 
The c = 0 case is the Shannon theoretical rate limit. The 
c = 1 case is called cutoff rate: the maximal rate with 
finite asymptotic width (f^ x~^dx = oo for c < 1). 
We will now find c for a general case. Derivation for 
simpler case: single BSC and test results can be found 

in im. 
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Figure 5. Experimental results for Method]^ TV = 1, 2,3,4, 5,6, 7, 8 required packets and received TVf = TV + 1 undamaged packets. The 
graphs show sorted widths for 1000 trials for length lOOOB messages (L = [SOOO/TV] steps). 


bit to be 


no-error growth 


expected growth 

(growth from Shannon's limit) 



before considering^ we have to expand 'wrong subtrees'-< up to weight J?. 


Figure 6. Schematic picture of the proper reconstruction we would like to find. After an error, we expand improper candidates until reaching 
the weight of the proper one. We need to find the expected size (U{w)) and probability distribution (V(it;)) for these subtrees of improper 
candidates. 


A. Finding the Pareto coefficient c 

Let us remind weights found in Section quanti¬ 
tatively describing how likely we will tolerate bit-flips 
depending on damage level of each packets (that i-th 
packet came through BSC). Denote error vector as 
where = 0 if fc-th bit of i-th 
packet agrees, Ef = 1 otherwise. Now the weight of 
fc-th block of candidate is IV(E^); 

M 

1L(E'=) =M-N + Y, kiPeiE,)) 

i=l 

where Pe(0) := 1 — e, Pe(l) := e. 

The total weight of a candidate is the sum of its W (E^). 
In every step of Method we choose the looking most 
probable node of the tree (candidate); having the largest 
weight W, and try to expand this node. 

The Shannon rate limit is ^ what 

is equivalent to Pr(E)lV(E) > 0. In other words, 
for rates below the Shannon limit, the weight of the 
proper candidates is on average growing. In contrast, the 
weights of improper candidates is on average decreasing, 
allowing to cut these branches of reconstruction tree. 

The situation is depicted in Fig. The dots 
correspond to length N bit blocks of the message. 
The black like represents the (unknown) proper 
reconstruction we are looking for. As long as there 
are no errors, the weight is growing (FL(Efc) > 0), 


hence it will remain the favorite candidate of sequential 
decoding. However, the weight drops while errors 
appear (IV(Efc) < 0). In this case, the decoder will 
try different (improper) candidates, until exploring all 
possibilities having higher weight than the weight of 
the proper candidate - ’’wrong subtrees”: triangles in 
the figure. Hence, what is crucial is the weight drop w 
parameter (arrows in the figure); the maximal weight on 
the proper path up to the considered position minus the 
current weight. Without error, the currently considered 
node has the highest weight so far: w = 0. However, 
larger w can theoretically also appear, with probability 
decreasing exponentially with w. Situation with w 
parameter requires considering subtrees of improper 
candidates of size growing exponentially with w. 

We will now And these two behaviors. Let us start 
with finding the expect number of improper nodes to 
expand for a given weight drop. Define U{w) as the 
expected number of nodes in subtree with w weight 
drop. If a given node is created, we potentially need to 
expand all of its 2^ children. Let us assume that each of 
them corresponds to a completely random E = {Ei}i 
vector: Pr(E) = 1/2^^. Finally, the expected number 
of nodes to expand for each of 2^ possibilities is 
EesJo i}“ U{w + W(E))/2^, getting functional equa¬ 
tion (Q. Large w behavior is crucial here - it corresponds 
to rare large subtrees of improper candidates. In this case 
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U{w) the expected number of improper nodes in subtree with w weight drop 
1 + 2^ Ee 6 { 0 .i}" U{w + W{E))/2^ for w > 0 
0 for w < 0 


U{w) = 


V (w) 


V{w) := Pr(weight on the correct path will drop by at most w) 

i SEe{o,i}“ (a=i PeAEi)) ■ Viw + fP(E)) for rc > 0 
1 0 for re < 0 


(7) 


(8) 


we can neglect the ”1+” term and the remaining linear 
functional equation should have asymptotic behavior of 
form: 

U(w) (X 2'^’" for w —>■ oo and some u > 0 
Inserting this assumption to (|^ we get: 

Ee{o,i}“ 

Ee{0,l}^ i 
M 

iv = M-vMa±(i^ 

it 

M 

iV = ^ 1 - /i„(ez) (9) 

where hu{e) = 1 is the Renyi entropy. 

Let us now hnd the probability distribution of values 
of w obtained while reconstruction. Dehne V{w) as 
CDF for w: the probability that weight on the cor¬ 
rect path will drop by at most w. Comparing situa¬ 
tion in successive positions and using BSC assumption: 
Pr(E) = nil P^^{Ei), we get (|^ functional equation. 
For w < 0, V{w) = 0. As discussed, 0 < C(0) is 
the probability that a proper node will have the highest 
weight so far. From dehnition, limij;_>,oo E(ii;) = 1. 
Finally, we can assume that the functional equation has 
solution of form: 


M 

lg(e,^- + (l-e.)i- 

V 
M 

N = Y,^-hl-v{e^) (10) 

i=l 

comparing with formula (|^ for u, we see that v = 1 — u. 


n = m-Y^ 


Flaving the exponent coefficients of U{w) and V(w) 
functions from (j^ and (10 1 , we can assume that U(w) « 
c„2““, V(w) « 1 — for some unknown coef- 

heients c„, c„. Now the asymptotic probability that the 
number of nodes of subtree of wrong correction for a 
given position will exceed some number of steps s is: 


Pr(:j(t nodes > s) « Pr f w > 


lg(s/c„) 


l-c„2-= 1-, 


— v/ h 


Pr{4f nodes in improper subtree >s)r:! 1 — CpS 

( 11 ) 

where c:=v/u=lju—1 (i.e. u = 1/(1 -f c)) and 
Cp = We have obtained the Pareto probability 

distribution with exponent which can be found analyti¬ 
cally: it is c if 

M 

E = ^ 1 — ^i/(i-i-c)(ei) 

i=l 

In other words, the sum of i?c(e) = 1 — hi/(i+c)(c) 
over all received bits should be the number of bits of 
the message. 


1 — V (w) oc 2 for w —>■ (X) and some i; > 0 

Inserting this assumption to ([^ we get: 

E \ i 

2v(M-N) ^ Y Y[PeAE^y~'' = 
EG{ 0 , 1 }" i 


These considerations were for unidirectional de¬ 
coding: in direction of encoding, starting from the 
initial_state. As discussed and tested in ifTTIl for single 
BSC, we could also perform bidirectional correction: 
simultaneously perform backward direction sequential 
decoding from the final_state. Failure of decoding is 
caused by critical error concentration (CEC): rare event 
requiring to consider very large number of improper 


^ 2-v{w+M-N+J2i'isiPiiEi))) 
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Figure 7. Experimental results (solid lines) and Pareto approximation (dashed line) for N = 5,6,7, 8 required packets, N — 1 received 
undamaged and two packets having damage level chosen to obtain c ^ 1 Pareto coefficient (e = 0.05 and 0.04 bit-flip probability). While 
they have the same c, larger N allows to stabilize the correction process. Left: sorted average width from 1000 trials. Right: the same plot in 
log-log scale for number of trial rescaled to [0,1] range. 



Figure 8. Experimental results (solid lines) and Pareto approximation (dashed lines) for N = 3 required packets with 2 undamaged and 
d = 6, 7, 8, 9 badly damaged packets: with e = 0.2 bit-flip probability. Left: sorted average width from 1000 trials. Right: the same plot in 
log-log scale for number of trial rescaled to [0,1] range. 


candidates (large weight drop w) - exceeding assumed 
resource limit. While single CEC prevents successful 
reconstruction for unidirectional decoding, bidirectional 
decoding can perform nearly complete reconstruction in 
this case (and mark the uncertain part). Two CECs are 
required to essentially cripple the process in this case; 
probability of such event is approximately square of 
probability of a single CEC. Einally, Pareto coefficient 
c practically doubles in this case: c = 1/2 bidirectional 
correction has similar behavior as c = 1 unidirectional. 

B. Experimental tests 

Extensive tests for single BSC, including bidirectional 
correction, can be found in Ha. We will now discuss 
results of two experiments for the general JRC case 
relying on using multiple packets. Implementation ifTSll 
were used for the tests (unidirectional), both tests use 
lOOOB messages. 

The first experiment, presented in Eigure uses 

— 1 undamaged packets and two damaged packets: 
of e = 0.04 and 0.05, chosen to get c « 1 (cutoff rate. 


'Yhi ~ Rl)- Increasing N does not affect c, but the 

graphs show some dependence on N: smaller N case 
has a bad behaving tail, stabilized thanks to using more 
nodes. 

The second experiment, presented in Eigure tests 
the exploitation of tiny informational content of highly 
damaged packets: of e = 0.2, there are d of such packets. 
This level of damage is difficult to handle with standard 
error correction, while here we see that these damaged 
packets can be used as a replacement for one missing 
undamaged packet. Increase of their number d essentially 
improves the correction process. The graphs show that 
the Pareto assumption is not in perfect agreement with 
experiment, however the found coefficients describe well 
the general behavior. 

C. Summary for the general case 

Let us summarize the theoretical part: obtaining M 
packets with correspondingly {ei}j=i,,M bit-flip proba¬ 
bility, we should And c such that N = R-ci^i) = 

J2iLi 1 “ ^i/(i+c)(e*)- 
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As hu{0) — 0 and generally /i > 0, if there is N or 
more undamaged packets, no such c can be found - we 
have the undamaged case with Gaussian distribution of 
weights (if receiving anything more than N undamaged 
packets). 

Otherwise, as long c > 0, we have approximately 
c Pareto distribution of weight. If c < 0, we are 
below the Shannon rate: there is definitely not sufficient 
information for reconstruction - we have to wait for more 
packets. For 0 < c < 1 the (average) weight for infinite 
packets would be infinite ( x~'^dx = oo for c < 1). 
However, finite length packets can be still handled in this 
range. Finally, the receiver should wait until reaching 
some arbitrarily chosen Cmin value: 

M 

(12) 

i=l 

then perform the first trial of reconstruction. If assumed 
resource limit (time and memory) was not sufficient, 
it should wait for another packet (or more), then try 
reconstruction one more time and so on. The successive 
trials can use partial reconstruction from the previous 
trials. 

There is a question of the minimal value (Cmin) to 
choose before the first trial. Optimally choosing this 
parameter for a given scenario is a difficult question, 
conditioned by value of packets and available resources. 
Generally Cmin & [1/2,1] is suggested for unidirectional 
correction, twice smaller for bidirectional. 

V. Unknown noise level problem and 

CONCATENATION WITH THE FOUNTAIN CODES 

JRC allows to combine the remaining informational 
content of damaged packets without the need of a priory 
knowledge of their final damage levels. However, at 
least for the discussed sequential decoding, efficient 
joint reconstruction is rather limited to a relatively small 
number of packets, what can be relaxed by interleaving. 

In contrast, FC, especially the Raptor codes, can 
easily operate on thousands of packets. They cannot 
reconstruct from damaged packets, what can be handled 
by concatenating them with FEC. However, it requires 
a priori knowledge of the final noise level for efficient 
operation. Observe that we could concatenate FC 
with JRC instead like in Fig. what should allow to 
combine their advantages: operate on a large number of 
packets and with incomplete or nor a priori knowledge 
of the final noise level. 

To better understand these three possibilities: 
FC+FEC, JRC and FC+JRC like in Eig. let as 
consider an idealized problem with minimal a priori 
knowledge of the final noise level. Eor example imagine 
the broadcasting situation: while every receiver and 



Figure 9. Three considered approaches to the unknown noise 
level (UNL) problem: FC+FEC, JRC, FC+JRC. While JRC can only 
reconstruct from a relatively small number of packets, FC can work 
with a very large number (we assume asymptotic setting). 

packet can have some (practically random) individual 
noise level, the sender needs to prepare universal 
packets for all receivers. Let us write this problem in a 
general setting: 

Unknown noise level problem (UNL): we have a 
large number of given type of channels, each having an 
independent random parameter from some probability 
distribution. The encoder knows the probability distribu¬ 
tion, not the individual values. The decoder knows the 
parameters for all channels. How to maximize the rate? 

Eor simplicity, let us focus on BSC channels with 
e € [0,1/2] bit-flip probability chosen with uniform 
probability distribution. 

We would expect to approach the rate for the situation 
when the damage levels are known: 

/•1/2 1 
h:=2 (1 - h(e)) de = l- —r « 0.27865 

Jo ln(4) 

Let us try to handle this UNL problem using the three 
approaches from Eig. For simplicity, let us assume 
that the used methods are perfect: FC has asymptotically 
p rate when p of packets are retrieved undamaged, FEC 
has rate 1 — h{e) for BSC with e bit-flip probability, 
JRC requires for reconstruction that sum of 1 — h{e) 
over received packets is at least the number of required 
undamaged packets (N). Additionally, we are assuming 
here that there are no directly lost packets, however, the 
not fully repaired packets are treated as lost (discarded). 

1) FC-hFEC: in this standard setting we need to 
choose redundancy level for the EECs: the largest e 
they are able to repair. While choosing it as e for all 
packets, statistically 2e of packets should be corrected 
for the assumed uniform probability e S [0,1/2]. The 
problem of missing packets (too damaged to be repaired) 
is handled by the FC: the sender produces ^ more 
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packets than required for reconstruction. Finally, the best FC can produce l/p^ packets, leading to asymptotically 
possible rate we could obtain in this approach is complete reconstruction. Finally, 


max 2e • (1 — h(e)) « 0.11712 for e « 0.15455 
se[o,i/ 2 ] 

Even for the idealized conditions, this approach is still 
far from the optimal rate. JRC allows to improve it. 


2) JRC: in pure JRC setting, we can choose two 
parameters: the minimal number of undamaged packets 
for reconstruction (N) and the number of actually sent 
packets (M), hence the rate is N/M. In the idealized 
situation, successful reconstruction is performed when 

M 

J2{l-h{e.))>N 

We are assuming that are chosen randomly, hence the 
above condition is fulhlled only with some probability. 
Approximate probability of fulhlling it for = 2 and 
M from 3 to 16 is succeedingly: 

0.013,0.063,0.161,0.297,0.45,0.593,0.721, 


0.817,0.885,0.931,0.961,0.978,0.988,0.994 


for example rate N/M = 0.25 allows to reconstruct 
only in « 0.593 of cases. This inevitable problem of 
not correcting in all random situations was handled in 
the FC+FEC case by assuming asymptotic number of 
packet case for EC, we will use it in the next: EC+JRC 
setting. 

Let us analyze asymptotic behavior of using pure JRC. 
Assuming uniform probability distribution e G [0,1/2], 
N = 1 — h{e) is a random variable with h « 0.27865 
expected value and 


a = 



V21 - 27r2 
61n(2) 


cr « 0.26999 standard deviation. M packets are suffi¬ 
cient for reconstruction with probability 


PriM, N)=Pr= Pr(Xi -f ^2 + ... + Xm > N) 


Using the central limit theorem, we get pr ~ 0.5 for 
large N « JiM, or equivalently rate equal N/M = h, 
what is the maximum for our problem. Eor a more prac¬ 
tical like 0.99, we have N « JiM — ay/M{p^), 
or in other words. 


rate is 


N 

M 


- (T-erf ^{pr) 

Vm 


approaching h for M —> oo. 


3) FC-hJRC As JRC can be efficiently used for a 
relatively low number of packets and we would like to 
achieve reconstmction of all packets, we can concatenate 
it with EC layer to handle the problem of < 1- The 


rate for a given N is max — Pr{M, N) 
mM 

As it is shown in Table [II] or can be deduced from 
the pure JRC case, the h rate limit is asymptotically 
approached. While N oo, we rely less on EC: Pr —1. 


Table II 

Parameters for JRC+FC and optimal choice of M 


N 

I 

2 

3 

4 

10 

too 

M 

5 

to 

15 

19 

45 

398 

Pr 

0.7222 

0.8167 

0.8715 

0.8646 

0.9228 

0.9800 

rate 

0.1444 

0.1633 

0.1743 

0.1820 

0.2051 

0.2462 


VI. Conclusions and further perspectives 

Joint Reconstmction Codes were introduced and dis¬ 
cussed as an enhancement of the fountain codes concept. 
JRC allows to combine two processes: of reconstruction 
from some subset of packets and of forward error 
correction. Standard approach in this case is using EC 
with packets protected by forward error correction, how¬ 
ever, it requires a priori knowledge of damage levels, 
which is limited or unavailable in many scenarios. In 
contrast, JRC allows to operate with the same rates 
having only a posteriori knowledge of damage levels, 
at cost of being limited to a relatively small number 
of packets for reconstruction. It searches the space of 
candidates of the message to get agreement with all the 
received packets, tolerating disagreements accordingly 
to the individual damage level of each packet, which 
is estimated a posteriori. Thanks of it, the sender needs 
only to ensure a high enough statistics of packets for 
the receiver, do not have to care about the protection 
level for each individual packet - they are simultaneously 
payload and redundancy, decoded as having the optimal 
protection level. Presented theoretical analysis allows to 
calculate the Pareto coefficient using Renyi entropy, ap¬ 
proximately describing the sequential decoding process. 

Some examples of applications: 

• broadcasting: the sender cannot take individual 
noise levels into consideration, but needs to produce 
universal packets - JRC still allows to achieve rates 
as if every packet was optimally protected, 

• protecting a data storage: allowing to reconstruct 
the content from some a priori unknown subset 
of storage media, having a history dependent hnal 
damage levels, 

• for reducing hardware cost and energy consump¬ 
tion of various networks by replacing correction 
applied by every node, into a single reconstruction 
performed by the receiver. 
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* for more robust watermarking; requiring some sub¬ 
set of frames or parts of a picture, with capture 
conditions unknown to the sender, 

• for communication with rapidly varying conditions, 
where the receiver has usually a better (a posteriori) 
knowledge about the actual damage level. 

The undamaged case shows that the presented 
approach may have also advantages as a direct 
replacement of FC for low number of packets. For 
example in the case of one excess packet (M = + 1), 

decoding needs to consider on average only twice more 
steps then encoding and is practically always successful. 
In contrast, FC often fails in this case (with « 50% 
probability). 

This article only introduces to this new coding 
paradigm - there have remained plenty of research ques¬ 
tions and possibilities for improvements and optimiza¬ 
tions depending on specific applications. 

The implemented and analyzed decoding is currently 
unidirectional, what could be essentially improved by 
using bidirectional sequential decoding, effectively dou¬ 
bling the Pareto coefficient ini. Another line of devel¬ 
opment is including more sophisticated types of errors 
in the considered space of candidates/corrections - the 
discussed approach can even handle the synchronization 
errors, like deletion channel M- It might be also 
beneficial to compare the discussed sequential decoding 
approach with different realizations of the JRC setting, 
like using Low Density Parity Check ini. 

Additional improvements of reconstruction efficiency 
can be made by optimizing the choice of transition 
function /, for example including grouping or order 
of the sent packets into consideration, especially when 
the sender is likely to receive most or all of the pack¬ 
ets. Specifically, it should be chosen such that the set 
{extract{f[x])}x is nearly a permutation, or generally 
has large Hamming distances for subsets of packet 
which are more likely to be used for reconstruction (for 
example: the first sent) - increasing the probability of 
straightforward decoding. The choice of the bit positions 
for packets also influences performance (they should be 
far from each other). Additionally, using a Pseudoran¬ 
dom Number Generator initialized with a cryptographic 
key to choose / would allow to simultaneously encrypt 
the message - the used key would be essential to perform 
reconstruction. 

The main limitation of the discussed sequential de¬ 
coding approach is a relatively small number of pack¬ 
ets to reconstruct from, while FC does not have such 
limitation. This limitation can be relaxed by using in¬ 
terleaving, which efficient application requires further 
work. Concatenation of JRCh-FC can be also beneficial, 
as discussed in Section 0 

Finally, this article assumes (a posteriori) knowledge 


of damage levels (e), what requires some estimation 
process. The redundancy of received damaged packets 
should itself allow for this purpose, suggesting that this 
estimation could be made as a part of decoding process. 
Specifically, we could start with some arbitrary values, 
then try to modify them accordingly to already decoded 
part. Effective exploitation and analysis of above possi¬ 
bilities requires further work. 
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